Immersive Virtual Environment Research Articles

Investigating the effect of guided inquiry on learners with different prior knowledge in immersive virtual environments

Investigating the effect of guided inquiry on learners with different prior knowledge in immersive virtual environments

AdaPIP: Adaptive picture-in-picture guidance for 360° film watching

Abstract360° videos enable viewers to watch freely from different directions but inevitably prevent them from perceiving all the helpful information. To mitigate this problem, picture-in-picture (PIP) guidance was proposed using preview windows to show regions of interest (ROIs) outside the current view range. We identify several drawbacks of this representation and propose a new method for 360° film watching called AdaPIP. AdaPIP enhances traditional PIP by adaptively arranging preview windows with changeable view ranges and sizes. In addition, AdaPIP incorporates the advantage of arrow-based guidance by presenting circular windows with arrows attached to them to help users locate the corresponding ROIs more efficiently. We also adapted AdaPIP and Outside-In to HMD-based immersive virtual reality environments to demonstrate the usability of PIP-guided approaches beyond 2D screens. Comprehensive user experiments on 2D screens, as well as in VR environments, indicate that AdaPIP is superior to alternative methods in terms of visual experiences while maintaining a comparable degree of immersion.

Different smartphone tasks and traffic complexity affect pedestrian awareness of co-existing road objects and cerebral oxygenation during shared space walking

Distraction, primarily owing to the widespread use of cell phones and mobile devices while walking, is a contributing factor to pedestrian injuries. Pedestrians distracted by smartphones exhibit decreased situational awareness, influenced by the nature of concurrent tasks. However, little is known about the neural correlations between smartphone use and pedestrian responses to safety–critical events in a walking environment. This study recruited 30 young adults to investigate pedestrians’ ability to selectively respond to road objects while using smartphones (texting and reading) while walking in an immersive virtual environment system. The level of traffic complexity for the road conditions was manipulated by varying the number of safety–critical road objects coexisting in the environment at any given time and the number of required responses to the number of objects. During each 7-min task condition, cerebral oxygenation was measured using functional near-infrared spectroscopy. Furthermore, smartphone task performance, response times to road objects, and perceived workload were also measured. The participants exhibited reduced texting speed and reading performance when responding sequentially to coexisting road objects. Compared with other road conditions, those requiring two responses to four objects resulted in an increased workload and longer object response times. Participants perceived a higher workload in conditions involving tasks with smartphones than those without smartphones. More importantly, increased concentrations of oxygenated hemoglobin in the prefrontal cortex were observed when participants were concurrently performing smartphone and pedestrian tasks. Compared to smartphone reading, smartphone texting induced greater cerebral oxygenation in the bilateral frontopolar cortex. These results suggest that continuously texting while walking may demand additional cognitive resources for executive function. The findings of this study represent an initial step towards elucidating the cognitive mechanisms underlying attentional impairments experienced during smartphone distraction and working towards a viable solution to mitigate pedestrian risk.

Assessing Learning in an Immersive Virtual Reality: A Curriculum-Based Experiment in Chemistry Education

Despite the recent advances in Virtual Reality technology and its use in education, the review of the literature shows several gaps in research on how immersive virtual environments impact the learning process. In particular, the lack of curriculum-specific experiments along with investigations of the effects of different content, activity, and interaction types in the current VR studies has been identified as a significant shortcoming. This has been more significant in STEM fields, where VR has the potential to offer engaging experiential learning opportunities. The study reported here was designed to address this gap by assessing the effect of authentic visualization and interaction types on learning a particular scientific concept. A use case scenario of “orbital hybridization” in chemistry education was selected to create this experiment and to collect data for analysis. We collected data on learning outcomes, task-completion efficiency, accuracy, and subjective usability. A combination of learning content and tasks designed based on the relevant educational theories was presented to three groups: 2D, VR interaction type 1 (hand gestures), and VR interaction type 2 (ray casting). The results showed that VR could improve learning and that interaction type could influence efficiency and accuracy depending on the task.

Multi-source point cloud registration for urban areas using a coarse-to-fine approach

ABSTRACT The foundational step in constructing a digital twin involves transforming the physical world into an immersive 3D virtual environment. Unmanned aerial vehicle (UAV)-based photogrammetry, mobile mapping system (MMS), and terrestrial laser scanning (TLS) serve as powerful tools for 3D reconstruction of urban environments. Given the distinctive strengths and weaknesses of each sensor, a multi-source point cloud registration technique is imperative to achieve a detailed and precisely geolocated 3D urban model. Hence, we developed coarse-to-fine approach methods, called building exterior wall-based (BEWB) and building outline-based (BOB) algorithms, to register point clouds captured by diverse sensors in urban scenes. The BEWB algorithm coarsely registers urban point clouds by extracting building exterior walls, establishing correspondence points, and effectively removing outliers within this correspondence point set. The BOB algorithm precisely registers the urban point clouds acquired from multiple sensors by leveraging building outlines and points corresponding to the ground of the point clouds. To validate the proposed algorithms, we initially registered a UAV-based photogrammetry point cloud with the MMS point cloud and subsequently registered the TLS point clouds. Comprehensive quantitative and qualitative analyzes of the results demonstrate that our algorithms outperform existing methods in achieving precise registration. By successfully registering three distinct point clouds, we generated a comprehensive urban scene point cloud characterized by enhanced precision, point density, and geolocation accuracy.

Speech intelligibility in reverberation based on audio-visual scenes recordings reproduced in a 3D virtual environment

Audio-visual scenes were collected in a medium-sized reverberant conference hall through in-field 3rd-order ambisonics impulse response recordings and 360-degree stereoscopic videos. The visual scenes included cues of the room and the location of the sound sources, without lip-sync-related cues. Speech intelligibility tests based on seven audio-visual scenes were administered inside an immersive virtual 3D environment reproduced through a spherical 16-speaker array synched with a head-mounted display. Forty normal-hearing subjects were engaged to test the effects on speech intelligibility of a talker in front of the listener and amplified by two lateral symmetrical loudspeakers, in the case of (i) different listener-to-talker distances, (ii) one-talker noise at various azimuth angles around the listener, (iii) high reverberation with –5 dB signal-to-noise ratio, (iv) self-motion, and (v) visual cues. We conducted tests in four configurations, that is, audio-visual and audio-only, both with self-motion and in the static condition. The static audio-only tests scored the highest speech intelligibility, followed by a tie between audio-visual with self-motion and in the static condition. Speech intelligibility decreased as the target-to-listener distance increased in all the noisy scenes. Additionally, speech intelligibility increased when the noise azimuth was at 120° compared to both 180° and 0° , with the talker at approximately 8 m from the listener. The advantage of the spatial separation of the noise signal in reverberation is evident in the case of the audio-visual with self-motion test. This suggests a spatial release from masking in the presence of reverberation, one-talker-interfering noise and within an more ecological scene.

Observing heroic behavior and its influencing factors in immersive virtual environments

Studying heroism in controlled settings presents challenges and ethical controversies due to its association with physical risk. Leveraging virtual reality (VR) technology, we conducted a three-study series with 397 participants from China to investigate heroic actions. Participants unexpectedly witnessed a criminal event in a simulated scenario, allowing observation of their tendency to physically intercept a thief. We examined situational factors (voluntariness, authority, and risk) and personal variables [gender, impulsivity, empathy, and social value orientation (SVO)] that may influence heroism. Also, the potential association between heroism and social conformity was explored. In terms of situational variables, voluntariness modulated participants' tendency to intercept the escaping thief, while perceived risk demonstrated its impact by interacting with gender. That is, in study 3 where the perceived risk was expected to be higher (as supported by an online study 5), males exhibited a greater inclination toward heroic behavior compared to females. Regarding other personal variables, the tendency to engage in heroic behavior decreased as empathy levels rose among males, whereas the opposite trend was observed for females. SVO influenced heroic behavior but without a gender interaction. Finally, an inverse relationship between heroism and social conformity was observed. The robustness of these findings was partly supported by the Chinese sample (but not the international sample) of an online study 4 that provided written descriptions of VR scenarios, indicating cultural variations. These results advance insights into motivational factors influencing heroism in the context of restoring order and highlight the power of VR technology in examining social psychological hypotheses beyond ethical constraints.

Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses

IntroductionDespite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users’ own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users’ experience regarding embodiment and their interaction with different functional levels.MethodsTo investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey.ResultsAmong the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented.ConclusionsOur work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.

Assessing Human Reactions in a Virtual Crowd Based on Crowd Disposition, Perceived Agency, and User Traits

Immersive virtual environments populated by real and virtual humans provide valuable insights into human decision-making processes under controlled conditions. Existing literature indicates elevated comfort, higher presence, and a more positive user experience when virtual humans exhibit rich behaviors. Based on this knowledge, we conducted a web-based, interactive study, in which participants were embodied within a virtual crowd with complex behaviors driven by an underlying psychological model. While participants interacted with a group of autonomous humanoid agents in a shopping scenario similar to Black Friday, the platform recorded their non-verbal behaviors. In this independent-subjects study, we investigated behavioral and emotional variances across participants with diverse backgrounds focusing on two conditions: perceived agency and the crowd’s emotional disposition. For perceived agency, one group of participants was told that the other crowd members were avatars controlled by humans, while another group was told that they were artificial agents. For emotional disposition, the crowd behaved either in a docile or hostile manner. The results suggest that the crowd’s disposition and specific participant traits significantly affected certain emotions and behaviors. For instance, participants collected fewer items and reported a higher increase of negative emotions when placed in a hostile crowd. However, perceived agency did not yield any statistically significant effects.

LNMVSNet: A Low-Noise Multi-View Stereo Depth Inference Method for 3D Reconstruction.

With the widespread adoption of modern RGB cameras, an abundance of RGB images is available everywhere. Therefore, multi-view stereo (MVS) 3D reconstruction has been extensively applied across various fields because of its cost-effectiveness and accessibility, which involves multi-view depth estimation and stereo matching algorithms. However, MVS tasks face noise challenges because of natural multiplicative noise and negative gain in algorithms, which reduce the quality and accuracy of the generated models and depth maps. Traditional MVS methods often struggle with noise, relying on assumptions that do not always hold true under real-world conditions, while deep learning-based MVS approaches tend to suffer from high noise sensitivity. To overcome these challenges, we introduce LNMVSNet, a deep learning network designed to enhance local feature attention and fuse features across different scales, aiming for low-noise, high-precision MVS 3D reconstruction. Through extensive evaluation of multiple benchmark datasets, LNMVSNet has demonstrated its superior performance, showcasing its ability to improve reconstruction accuracy and completeness, especially in the recovery of fine details and clear feature delineation. This advancement brings hope for the widespread application of MVS, ranging from precise industrial part inspection to the creation of immersive virtual environments.

Immersive virtual environments as a tool to improve confidence and role expectancy in prospective social work students: a proof-of-concept study

ABSTRACT This paper explores the use of immersive technology in social work education, specifically through a virtual simulation called ‘Branch-VR’ designed for young people taking part in a Widening Participation Programme for entry into social work training. The study involved 24 participants aged 17–18, who interacted with a branching narrative simulation of a social work home visit, making decisions that influenced the outcome. Utilising a mixed-methods quasi-experimental design, the paper examines participants’ attitudes toward using immersive technology as a training tool, its impact on their role expectations when engaging with service users in a home environment, and its influence on their confidence in pursuing a career in social work. The findings suggest that immersive simulations like Branch-VR can bridge the gap between theoretical knowledge and practical application. This approach also boosts learners’ confidence (W = 196.000, Z = 2.808, p = .005), providing a safe environment for experiential learning and decision-making. The paper underscores the importance of integrating immersive technology with traditional training methods, emphasizing its role as an augmentation to foundational social work education.

Immersion in a relaxing virtual reality environment is associated with similar effects on stress and anxiety as heart rate variability biofeedback

Practicing guided breathing at 0.1 Hz in virtual reality yields psychological and physiological benefits. Nonetheless, it remains uncertain whether these effects surpass those induced in a real-world setting. Indeed, the potential influence of the virtual environment on perceived stress and anxiety is not yet fully understood. In this experiment, we aimed to compare the effects of heart rate variability biofeedback combining both haptic and visual cues in real and virtual reality settings among the same group of participants. Additionally, to discern whether the psychological benefits arise from viewing an environment in virtual reality or from the act of performing guided breathing in this specific setting, a “control” immersion condition was introduced. 36 healthy sport students (9 females) participated in this study, performing both the real and virtual reality protocols in a randomized order. Anxiety and stress levels were assessed using the STAI-Y questionnaire and a visual analog scale, respectively. Physiological effects were assessed through measures of heart rate variability, and the performance of cardiac coherence was compared between the real and virtual implementations of guided breathing. As expected, both real and virtual reality heart rate variability biofeedback led to similar physiological modulations and cardiac coherence performances. A decrease in stress and anxiety was observed in both protocols, particularly among participants who initially reported higher stress or anxiety levels. However, no additional changes in psychological states were observed when performing guided breathing while immersed in the virtual environment.

IVR-fNIRS: studying brain functions in a fully immersive virtual environment.

Immersive virtual reality (iVR) employs head-mounted displays or cave-like environments to create a sensory-rich virtual experience that simulates the physical presence of a user in a digital space. The technology holds immense promise in neuroscience research and therapy. In particular, virtual reality (VR) technologies facilitate the development of diverse tasks and scenarios closely mirroring real-life situations to stimulate the brain within a controlled and secure setting. It also offers a cost-effective solution in providing a similar sense of interaction to users when conventional stimulation methods are limited or unfeasible. Although combining iVR with traditional brain imaging techniques may be difficult due to signal interference or instrumental issues, recent work has proposed the use of functional near infrared spectroscopy (fNIRS) in conjunction with iVR for versatile brain stimulation paradigms and flexible examination of brain responses. We present a comprehensive review of current research studies employing an iVR-fNIRS setup, covering device types, stimulation approaches, data analysis methods, and major scientific findings. The literature demonstrates a high potential for iVR-fNIRS to explore various types of cognitive, behavioral, and motor functions in a fully immersive VR (iVR) environment. Such studies should set a foundation for adaptive iVR programs for both training (e.g., in novel environments) and clinical therapeutics (e.g., pain, motor and sensory disorders and other psychiatric conditions).

An Immersive Virtual Learning Environment for the Development of Hard Skills: A Scientometric Analysis and Systematic Review

A scientometric analysis and systematic review of an immersive virtual learning environment (IVLE) designed to improve the development of hard skills comprised this study. The review focused on the period between 2016 and 2023 and was conducted using the Scopus database. This study employed a scientometrics analysis and systematic review methodology. In this study, a comprehensive analysis was conducted on a corpus of 12 scholarly articles that have been published within the last 8 years. This study looked at the aggregate magnitude and trajectory of scholarly publications and looked at a number of trends in scholarly research, including annual reports, article counts, article distribution across different sources, finding the most useful sources, keyword analysis, looking into the best collaboration groups, looking at how themes have changed over time, judging contributions, and practical implications. The primary findings regarding the data and document types indicated that a total of 12 articles were published across 10 different sources within the timeframe of 2016 to 2023. The research conducted on IVLE for the development of hard skills exhibited fluctuations in its pace, alternating between deceleration and positive acceleration. The growth rate of the articles peaked in 2022. The primary publication with the greatest of source is “Lecture Notes in Networks and Systems.” Keyword Plus growth rate for terms like “virtual reality,” “e-learning,” and “teaching” The nations exhibit a high degree of collaboration in the field of research. The findings indicated that the authors hailing from Mexico and Colombia exhibited the greatest frequency (n = 2). The thematic evolution of the study indicated two significant advancements: firstly, the emergence of e-learning studies that have propelled the field of e-learning, and secondly, the integration of virtual reality into e-learning practices. The classification of applications encompasses various components: title, solution, learning strategy, and learning tasks.

Exploratory study of the role of outgroup emotional display on anger and emergency intentional helping behaviour in an immersive virtual environment.

Exploratory study of the role of outgroup emotional display on anger and emergency intentional helping behaviour in an immersive virtual environment.

Campus Culture Construction of Colleges and Universities Based on Virtual Reality Technology

This paper presents a novel approach to campus culture construction in colleges and universities utilizing virtual reality (VR) technology enhanced by Cartesian point Probabilistic LSTM (CPP-LSTM). The study aims to enhance campus culture by creating immersive virtual environments that promote engagement, collaboration, and learning among students and faculty members. Through simulated experiments and empirical validations, the efficacy of the CPP-LSTM-enhanced VR technology is evaluated in creating realistic and interactive virtual campus environments. Results demonstrate significant improvements in user engagement and satisfaction compared to traditional methods. For instance, users interacting with the CPP-LSTM-enhanced VR platform reported a 30% increase in perceived sense of belonging and a 25% improvement in overall satisfaction ratings. Additionally, the CPP-LSTM model enables the generation of personalized virtual experiences based on user preferences and feedback, leading to more immersive and impactful campus culture construction. These findings underscore the potential of VR technology with CPP-LSTM in revolutionizing campus culture construction and fostering a vibrant and inclusive campus community.

Behavioral intention, perception and user assessment in an immersive virtual reality environment with CFD simulations

This study explores technology acceptance, perception and user assessment of an immersive virtual reality environment with computational fluid dynamics simulations in engineering education. 57 participants from three different institutions tested the virtual reality application. Partial least squares structural equation modeling and interferential statistics were performed to predict and assess interrelations among constructs. Results show that the learning value, content value, intrinsic motivation and personal innovativeness are underlying factors behind students’ intention to use virtual reality. Pair-wise analysis indicates that users’ perceptions matter and positively affect their attitudes. In addition, the virtual reality application helps students perform significantly better in the post-knowledge test. Findings also highlight that prior experience and interest can affect students’ attitudes and behavioral intentions to accept the virtual reality application in education. Our study can guide lecturers and developers to achieve on-target immersive virtual reality learning environments in higher education.

Relative cue precision and prior knowledge contribute to the preference of proximal and distal landmarks in human orientation

A prevailing argument posits that distal landmarks dominate over proximal landmarks as orientation cues. However, no studies have tested this argument or examined the underlying mechanisms. This project aimed to close this gap by examining the roles of relative cue precision and prior knowledge in cue preference. Participants learned object locations with proximal and distal landmarks in an immersive virtual environment. After walking a path without seeing objects or landmarks, participants disoriented themselves by spinning in place and pointed to the objects with the reappearance of a proximal landmark being rotated −50°, a distal landmark being rotated 50°, or both (Conflict). Heading errors were examined. Experiment 1 manipulated the relative cue precision. Results showed that in Conflict condition, the observed weight on the distal cue (exceeding 0.5) changed with but remained higher than the weight predicted by the relative cue precision. This indicates that besides the relative cue precision, prior knowledge of distal cue dominance also influences orientation cue usage. In Experiments 2 and 3, participants walked a path stopping at one object location. Participants were informed of it explicitly in Experiment 2 but not in Experiment 3. Results showed that distal cue dominance still occurred in Experiment 3. However, in Experiment 2, proximal cue dominance appeared, and it was not predicted by the relative cue precision. These results suggest that prior knowledge of proximal cue dominance might have been invoked by the instruction of locations. Consistent with the Bayesian inference model, human cue usage in orientation is determined by relative cue precision and prior knowledge. The choice of prior knowledge can be influenced by instructions.

A virtual reality environment for supporting mental wellbeing of students on remote clinical placement: A multi-methods evaluation

BackgroundNursing and Allied Health Profession (NAHP) students undertake clinical placements as part of their pre-registration training. The remote nature of some placement sites, shiftwork and the emotionally challenging nature of the workload has led to mental wellbeing issues in many students. AimThis project aimed to evaluate a novel 3D immersive virtual reality environment that supports mental wellbeing for NAHP students on clinical placement. It comprises a calming 3D tropical beach environment where students and tutors can meet for reflection and mutual support. DesignA multi-methods design gathered quantitative impact data with validated measurement tools and qualitative output related to the lived experience of students. Settings and participantsAll 600 pre-registration NAHP students within the institution undertaking clinical placements were invited to participate, irrespective of mental wellbeing status. Students were randomly assigned to either a VR or Conventional cohort; all participants received the control support mechanism in a subsequent placement. MethodsAll participants completed an initial demographic and Readiness for Therapy survey followed by weekly Beck Anxiety and Depression Inventories during placement. All participants were invited to a semi-structured interview. ResultsOverall, 32 participants engaged with the application; although the VR cohort demonstrated improved scores on both Beck inventories, these were not statistically significant. This is probably due to the low response rate for the control cohort. A total of 15 interviews were conducted and several themes emerged from the data in relation.to both experiential outcomes (escapism, anonymity and sense of community) and instrumental outcomes (calming, mindfulness and combatting loneliness). ConclusionsUser feedback indicates that a VR environment can provide a calming escape from the pressures and anxiety arising from clinical placement for healthcare students. The relaxing beach environment facilitated mindfulness meditation and the additional opportunities for pseudo-anonymous interactions with peers and tutors were wellreceived by students.

Unveiling metaverse sentiments using machine learning approaches

PurposeThe metaverse, which is now revolutionizing how brands strategize their business needs, necessitates understanding individual opinions. Sentiment analysis deciphers emotions and uncovers a deeper understanding of user opinions and trends within this digital realm. Further, sentiments signify the underlying factor that triggers one’s intent to use technology like the metaverse. Positive sentiments often correlate with positive user experiences, while negative sentiments may signify issues or frustrations. Brands may consider these sentiments and implement them on their metaverse platforms for a seamless user experience.Design/methodology/approachThe current study adopts machine learning sentiment analysis techniques using Support Vector Machine, Doc2Vec, RNN, and CNN to explore the sentiment of individuals toward metaverse in a user-generated context. The topics were discovered using the topic modeling method, and sentiment analysis was performed subsequently.FindingsThe results revealed that the users had a positive notion about the experience and orientation of the metaverse while having a negative attitude towards the economy, data, and cyber security. The accuracy of each model has been analyzed, and it has been concluded that CNN provides better accuracy on an average of 89% compared to the other models.Research limitations/implicationsAnalyzing sentiment can reveal how the general public perceives the metaverse. Positive sentiment may suggest enthusiasm and readiness for adoption, while negative sentiment might indicate skepticism or concerns. Given the positive user notions about the metaverse’s experience and orientation, developers should continue to focus on creating innovative and immersive virtual environments. At the same time, users' concerns about data, cybersecurity and the economy are critical. The negative attitude toward the metaverse’s economy suggests a need for innovation in economic models within the metaverse. Also, developers and platform operators should prioritize robust data security measures. Implementing strong encryption and two-factor authentication and educating users about cybersecurity best practices can address these concerns and enhance user trust.Social implicationsIn terms of societal dynamics, the metaverse could revolutionize communication and relationships by altering traditional notions of proximity and the presence of its users. Further, virtual economies might emerge, with virtual assets having real-world value, presenting both opportunities and challenges for industries and regulators.Originality/valueThe current study contributes to research as it is the first of its kind to explore the sentiments of individuals toward the metaverse using deep learning techniques and evaluate the accuracy of these models.